Skip to content

resolve the prometheus discovery issue of same service‘s multiple instances in v3.8.3 - #3540

Open
deanwong wants to merge 1 commit into
openimsdk:release-v3.8.3from
deanwong:release-v3.8.3
Open

resolve the prometheus discovery issue of same service‘s multiple instances in v3.8.3#3540
deanwong wants to merge 1 commit into
openimsdk:release-v3.8.3from
deanwong:release-v3.8.3

Conversation

@deanwong

@deanwong deanwong commented Aug 27, 2025

Copy link
Copy Markdown

Fixes #3073

Description
When setAutoPorts enabled, if starting two rpc service (e.g. user rpc service) , the second one will overwrite the etcd value with same key, the code in start.go as below

if autoSetPorts {
	listener, prometheusPort, err = getAutoPort()
	if err != nil {
		return err
	}

	etcdClient := client.(*etcd.SvcDiscoveryRegistryImpl).GetClient()
        // this step will overwrtite the same service with different value
	_, err = etcdClient.Put(ctx, prommetrics.BuildDiscoveryKey(rpcRegisterName), jsonutil.StructToJsonString(prommetrics.BuildDefaultTarget(registerIP, prometheusPort)))
	if err != nil {
		return errs.WrapMsg(err, "etcd put err")
	}
}

For example, I started user service, the etcd value like

openim/prometheus_discovery/user {"target":"10.102.22.175:40145","labels":{"namespace":"default"}}

if I started another user service in different node, the etcd value was covered

openim/prometheus_discovery/user {"target":"/168.64.9.42:34657","labels":{"namespace":"default"}}

Fixed
Add nodeId and port in etcd key, the value will be like this in same scenario

openim/prometheus_discovery/user/10.102.22.175:40145 {"target":"10.102.22.175:40145","labels":{"namespace":"default"}}
openim/prometheus_discovery/user/168.64.9.42:34657 {"target":"/168.64.9.42:34657","labels":{"namespace":"default"}}

Moreover, the prometheus discovery will return correct node instances,

[
    {
        "targets": [
            "10.102.22.175:40145",
            "168.64.9.42:34657"
        ],
        "labels": {
            "namespace": "default"
        }
    }
]

@pull-request-size pull-request-size Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Aug 27, 2025

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Bloomingg Bloomingg Awaiting requested review from Bloomingg
@FGadvancer FGadvancer Awaiting requested review from FGadvancer FGadvancer is a code owner
@skiffer-git skiffer-git Awaiting requested review from skiffer-git skiffer-git is a code owner
@withchao withchao Awaiting requested review from withchao withchao is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@deanwong